I’ll wait for the Gaudi integration test CI to pass before merging anything:
https://github.com/huggingface/text-generation-inference/actions/runs/14591230970/job/40927197928?pr=3160

The previous run was green, which gives me confidence in the current changes:
https://github.com/huggingface/text-generation-inference/actions/runs/14384130453/job/40336095297

Unfortunately, it can take days to get assigned a Gaudi1 runner 😭, so I figured I could start iterating on your reviews in the meantime rather than wait for the CI to finish before requesting feedback. In any case, I’ll only merge once the Gaudi integration test passes in the CI also

Ok, I will wait for the new runners before adding Gaudi to the CI, as indeed the DL1 runners are super unreliable

The runners for Gaudi are ready! 🙌 Thanks @regisss

Just requesting some new reviews to be sure everything is still okay. Since the last review I just rebased on main and use the new runners. Now the integration test are passing and the runners are super fast! https://github.com/huggingface/text-generation-inference/actions/runs/15160963395/job/42627380206?pr=3160

@baptistecolle A couple of questions:

• It's not possible to select a specific runner for each test config right?
• If I want to add a new model to test, I just need to add a new test config in test_gaudi_generate.py?

@baptistecolle A couple of questions:

• It's not possible to select a specific runner for each test config right?
• If I want to add a new model to test, I just need to add a new test config in test_gaudi_generate.py?

1. No it is not i think this would require some rework of the build workflow which is global for all the hardwares. The best alternative would be to use a runner with 8 card and then set HABANA_VISIBLE_DEVICES=1
2. Yes, that's correct.

Some additional useful remark: you also need to add the new config with "run_by_default": True